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FOREWORD 


The  research  presented  in  this  report  was  conducted  under  the 
Aircrew  Performance  Project  in  the  Flight  Training  and  Aviator 
Selection  Technical  Area  of  the  Fort  Rucker  Field  Unit  of  the  Army 
Research  Institute  for  the  Behavioral  and  Social  Sciences  (ARI) . 

The  Aircrew  Performance  Project  provides  aircrew  training  research 
for  Army  Aviation  and  directly  supports  training  and  personnel 
research  needs  of  the  US  Army  Aviation  Center,  Fort  Rucker,  AL. 

One  program  of  research  is  the  investigation  of  tests  of  human 
performance  to  measure  information  processing,  psychomotor,  and 
time-sharing  skills  as  predictors  of  success  in  training  and  as 
Army  aviators.  ARI  Technical  Report  412  is  the  first  publication 
from  this  program  of  research. 

Related,  ongoing  programs  of  research  in  the  Flight  Training 
and  Aviator  Selection  Area  include  revision  of  the  existing  paper 
and  pencil  Flight  Aptitude  Selection  Test  (FAST)  and  the  development 
of  tests  and  methods  to  classify  aviator  trainees  for  mission 
specialities  in  advanced  undergraduate  training.  The  human  performance 
testing  research  is  also  related  to  the  job-sample  testing  application 
of  flight  simulators  in  the  Performance-Based  Aviation  Applicant 
Selection  System  program  in  the  Flight  Simulation  Area. 

The  results  of  the  human  performance  and  related  test  research 
are  expected  to  yield  a  battery  of  selection  and  classification 
tests  which  will  improve  the  validity  of  the  aviator  selection  and 
classification  processes.  It  is  also  anticipated  that  selected 
human  performance  tests  may  be  used  as  blocking  variables  or  co¬ 
variates  to  improve  the  statistical- power  and  efficiency  of  ARI 
aviator  training  experiments.  Future  research  with  the  human  per¬ 
formance  tests  will  Include  determination  of  major  factors  in  the 
research  battery,  the  development  of  selection  and  classification 
criteria,  and  the  assessment  of  predictive  validities  for  each  of 
the  applications. 

The  present  research  was  conducted  by  personnel  of  the  Flight 
Training  and  Aviator  Selection  Area  as  an  in-house  project  under 
Army  Project  2Q763743A772.  An  earlier  version  of  this  report  has 
been  printed  in  the  1978  Proceedings  of  the  Military  Testing 
Association. 
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OF  PERFORMANCE  IN  A  VISUAL-TRACKING  TASK 


BRIEF 


Requirement : 

The  US  Army  Aviation  Center  has  requested  that  the  US  Army  Research 
Institute  (ARI)  develop  testing  methods  to  screen  potential  Initial  Entry 
Rotary  Wing  (IERW)  Course  failures,  resignations,  and  poor  learners  not 
screened  by  the  existing  paper  and  pencil  Flight  Aptitude  Selection  Test 
(FAST).  Data  from  these  testing  methods  should  also  support  the  assignment 
of  students  in  IERW  to  mission-tracks  for  advanced  IERW  training. 

Procedure: 

The  purpose  of  this  investigation  was  to  evaluate  possible  sources 
of  confounding  in  the  results  of  a  psychomotor  test  of  ability  to  control 
an  unstable  system  prior  to  a  validation  study.  Recommended  changes  in 
testing  and  scoring  procedures  were  derived  from  a  review  of  related 
literature.  To  test  the  proposed  changes,  the  revised  test  was  administered 
to  nine  individuals  who  had  recently  resigned  or  been  eliminated  from 
warrant  officer  or  helicopter  pilot  training  and  20  of  their  contemporaries 
who  were  still  in  helicopter  pilot  training  at  the  US  Army  Aviation  Center. 

Findings: 

The  revised  testing  and  scoring  procedures  were  found  to  result  in 
better  information  about  categories  of  subject  performance  than  existing 
procedures. 

Application  of  Findings: 

The  findings  of  this  study  were  used  as  justification  for  more 
testing  time  with  the  psychomotor  test  in  planning  the  validation  study. 

An  additional  product  was  a  computer  program,  in  FORTRAN,  for  scoring 
individual  performance  data  from  the  psychomotor  test. 
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INTRODUCTION 


The  Army  Research  Institute  Field  Unit  at  the  Army  Aviation  Center 
is  conducting  aviator  trainee  selection  research  on  job-sample,  psychomotor, 
information  processing,  and  time-sharing  tests  to  improve  the  methods  of 
selecting  applicants  for  Army  helicopter  pilot  training.  This  paper  presents 
preliminary  results  from  an  investigation  of  methods  to  improve  the  measure¬ 
ment  of  visual  tracking  and  time-sharing  skill  as  a  part  of  that  research. 

In  this  section,  the  test  is  described,  some  sources  of  confounding  are 
considered  and  methods  to  overcome  the  confounding  are  presented.  Following 
the  introduction,  procedures  are  described  for  collecting  data  to  test 
selected  hypotheses  about  confounding.  Then,  the  results  of  the  data  col¬ 
lection  are  presented  and  the  discussion  section  focuses  on  the  prospects 
for  employing  data  from  the  visual  tracking  tests  in  time-sharing  and 
aviator  trainee  selection  research. 

Visual  Tracking  Test 

The  visual  tracking  test  used  in  the  current  research  was  designed 
to  measure  an  individual's  ability  to  control  an  unstable  system.  The 
test  device  is  a  single  axis,  compensatory  visual  tracking  task  described 
in  Pew,  Rollins,  Adams  and  Gray  (1977).  The  operator's  task  is  to  try  to 
maintain  a  light  spot  in  the  center  of  a  horizontal  display  using  lateral 
movements  of  a  finger  operated  joy-stick. 

The  test  difficulty  is  controlled  by  the  system  time  constant  in  the 
periodic  processing  of  the  control  stick  signal.  The  system  time  constant 
is  a  weighting  function  which  determines  the  rate  of  change  of  light  spot 
location  in  relation  to  control  stick  movements.  The  system  time  constant 
operates  as  a  divisor  so  that  the  size  of  the  constant  is  inversely  related 
to  test  difficulty.  The  test  device  periodically  samples  the  control  stick 
signal  and  computes  the  location  of  the  light  spot  as  a  weighted  function 
of  the  present  control  input  and  a  residual  component  from  previous  control 
signals  added  to  the  present  light  spot  location  value.  The  residual  compon¬ 
ent  is  correlated  with  the  operator's  previous  control  behaviors  and  greatly 
increases  the  difficulty  of  learning  effective  control  of  the  light  spot. 

The  tracking  test  device  can  be  operated  in  two  difficulty  modes: 
critical  and  fixed  difficulty  tracking.  The  fixed  difficulty,  or  fixed 
tracking  mode  was  designed  primarily  for  time-sharing  applications.  In  this 
mode,  the  tester  fixes  the  time  constant  at  a  given  value  and  the  operator 
performs  for  a  fixed  period  of  time.  The  measure  of  skill  in  fixed  tracking 
mode  is  the  total  absolute  deviation  of  the  light  spot  from  the  center  of 
the  display,'  averaged  across  the  time  of  performance. 


The  critical  difficulty,  or  critical  tracking  mode  is  used  to  estimate 
the  operator's  effective  time  delay.  The  effective  time  delay  represents 
the  minimum  operator  response  time  for  the  detection  and  correction  of 
errors  in  continuous  control  tasks  and  is  used  as  a  parameter  in  human 
information  processing  and  optimum  control  theory  models  of  operator 
behavior.  Operationally,  the  effective  time  delay  is  an  index  of  the 
amount  of  time  required  for  the  operator  to  detect  an  error  and  to  convert 
information  about  that  error  into  a  precise  control  movement.  Estimates 
of  the  effective  time  delay  from  the  critical  tracking  mode  are  employed 
as  the  value  of  the  fixed  time  constant  in  the  fixed  tracking  mode. 

To  measure  the  effective  time  delay  in  critical  tracking  mode,  the 
test  device  progressively  increases  test  difficulty  as  a  function  of  time 
in  the  performance.  Difficulty  is  progressively  increased  by  systemati¬ 
cally  reducing  the  size  of  the  time  constant  as  a  function  of  time  in 
performance.  As  the  time  constant  grows  smaller,  the  rate  of  change  in 
light  spot  location  per  unit  time  increases.  Eventually,  the  rate  of 
change  in  light  spot  location  becomes  so  rapid  that  the  operator  is 
unable  to  maintain  effective  control,  the  location  exceeds  the  limits  of 
the  display,  and  the  performance  ends.  The  measure  of  skill  is  the  esti¬ 
mated  effective  time  delay  which  is  the  size  of  the  system  time  constant 
at  the  end  of  the  performance.  This  investigation  was  designed  to 
evaluate  possible  confounding  effects  in  the  measurement  of  critical 
tracking  skill,  i.e.,  measurement  of  the  effective  time  delay. 

Confounding  Effects 

A  review  of  recent  research  with  the  present  test  (Pew  et  al.,  1977) 
and  two  similar  visual  tracking  tests  (Damos,  1977;  Gopher  &  North,  1974; 
North,  1977;  North,  Harris  &  Owens,  1978)  suggested  that  the  testing 
procedures  had  resulted  in  a  confounding  of  other  performance  factors 
with  the  measurement  of  visual  tracking  skill.  Pew  et  al.  defended  their 
procedures  with  evidence  of  test-retest  reliability  (Rose,  1974). 

In  the  research  with  similar  tests  there  was  evidence  that  confounding 
effects  had  degraded  the  validity  of  the  visual  tracking  data  to  estimate 
time-sharing  capacity  and  would  probably  degrade  the  validity  of  these 
measures  in  aviator  selection  decisions.  Gopher  et  al.  (1974)  and  North 
(1977)  observed  improvements  in  time-sharing  performance  as  contrasted 
with  predictions  from  single-task  performance.  Gopher  et  al.  offered 
three  hypotheses  which  might  account  for  these  discrepancies:  (a)  Use 
of  adaptive  logic  did  not  accurately  estimate  single-task  tracking  skill; 
(b)  There  was  an  improvement  of  single-task  tracking  skill  as  a  function 
of  practice  in  the  time-sharing  test;  and  (c)  There  is  an  independent 
time-sharing  skill  which  is  learned  only  in  practice  with  time-sharing 
tests.  At  the  conclusion  of  his  report.  North  (1977)  suggested  that 
"isolation  of  improvement  factors  is  an  important  direction  for  further 
research"  (p.  92). 
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Two  investigations  addressed  the  question  of  confounding  sources. 

In  a  transfer  of  training  experiment,  Damos  (1977)  found  weak  evidence 
of  improvement  of  both  single-task  and  time-sharing  skill  as  a  function 
of  practice  in  multiple-task  performance.  Indications  of  confounding 
effects  in  the  Damos  (1977)  data  were:  (a)  operator  unreliability  as 
evidenced  by  heterogeniety  of  variance;  and  (b)  failure  of  16.7%  of  the 
subjects,  8  of  48,  to  achieve  minimum  criterion  in  subsequent  time-sharing 
practice. 

Although  not  specifically  addressed  by  the  authors,  some  difficulties 
with  the  use  of  adaptive  logic  to  determine  test  difficulty  were  apparent 
in  the  investigation  of  test-retest  reliability  by  North  et  al.  (1978). 

The  adaptive  logic  was  used  to  establish  tracking  test  difficulty  in  the 
first  part  of  two  daily  testing  sessions.  After  fixing  the  level  of 
difficulty,  the  mean  root-mean-square  (RMS)  tracking  error  was  computed 
as  the  baseline  for  feedback  on  tracking  performance  in  the  time-sharing 
tests.  Table  1  is  a  summary  of  correlations  among  the  tracking  task 
difficulty  and  RMS  tracking  error  scores  across  the  two  daily  sessions 
and  two  days  of  testing. 

It  is  apparent  from  the  data  in  Table  1  that  test  difficulty  corre¬ 
lates  negatively  with  dual-task  RMS  tracking  error.  This  has  potentially 
serious  consequences  in  aviator  trainee  selection  research  because 
individuals  who  invest  greater  effort,  and  thus  achieve  higher  levels  of 
difficulty,  would  have  greater  difficulty  demonstrating  higher  levels  of 
time-sharing  capacity.  Conversely,  individuals  with  low  effort  in  the 
test  difficulty  phase  would  more  easily  exhibit  greater  capacity  in  time¬ 
sharing.  In  addition.  Table  1  shows  a  significant  decrease  of  correla¬ 
tion  between  single-task  and  time-sharing  RMS  error  between  the  first 
and  second  days  of  testing.  Since  the  high  test-retest  correlation 
(r  =  .90)  between  test  difficulty  across  the  two  days  of  testing  shows 
that  the  subjects  were  consistent  in  the  amount  of  effort  invested  in 
the  measurement  of  test  difficulty,  there  were  differential  changes 
among  individual  RMS  error  performances  as  a  function  of  changes  in 
single-task  performance.  This  is  supported  by  the  low  reliability  in 
single  task  RMS  performance  (rs  =  .01  &  .34)  and  the  moderate  test- 
retest  reliabilities  of  RMS  dual-task  performance  (jts  =  .49  &  .69). 

Therefore,  the  available  evidence  suggests  that  procedures  for 
measuring  task  difficulty  allow  for  two  major  sources  of  confounding: 

(a)  failure  to  train  to  asymptote  before  measuring  single-task  achieve¬ 
ment,  and  (b)  using  current  performance  error  as  a  criterion  for  adaptive 
adjustments  of  test  difficulty.  The  first  source  of  confounding  could 
apparently  be  removed  by  training  to  asymptote  or  by  developing  a  statis¬ 
tical  model  which  accurately  predicts  asymptotic  level  of  achievement 
from  selected  observations  of  learning  performance.  To  remove  the  second 
source  of  confounding  it  was  necessary  to  explain  how  differences  in 
individual  goals,  effort,  motivation  and  the  like  might  interact  with 
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jj  RMS  Single-Task 
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.34 

|  RMS  Dual-Task 
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i 


aNorth  et  al.,  (1978),  p.  16 

^Probability  is  less  than  .05  that  the  absolute  value  of  any  correlation 
greater  than  .388  is  greater  than  zero;  t/.388)  =  2.064,  df  =  24. 

Probability  is  less  than  .05  that  the  differences  between  each  pair  of 
Day  1  minus  Day  2  values  is  greater  than  zero;  Z^.52)  -  2^(.13)  =  2.14 
(Fisher's  r  to  Z  transform). 
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single-task  difficulty  to  obscure  level  of  achievement  and  then  to  provide 
a  means  of  measuring  the  degree  of  the  interaction  in  an  individual’s 
tracking  test  data.  As  suggested  in  the  following  discussion,  an  adequate 
solution  to  the  degree  of  effort  problem  is  necessary  to  improve  the 
validity  of  forced  as  well  as  adaptive  difficulty  testing  paradigms. 

Tolerance  for  Error 

In  a  review  of  human  performance  limitations  in  visual  tracking 
tasks,  Poulton  (1969)  uses  "tolerance  for  error"  to  explain  how  individual 
effort  interacts  with  measures  of  tracking  task  ability.  When  first 
introduced  to  a  relatively  easy  task,  i.e.,  one  with  a  single  dimension 
or  a  simple  control  system,  Poulton  says  that  initially  the  operator  will 
be  challenged  and  interested  in  the  task  giving  considerable  attention 
and  effort  to  task  performance.  Poulton  continues: 

But... [the  operator]  soon  discovers  what  he  can  and  cannot 
achieve,  and  settles  down  to  give  what  he  considers  to  be 
an  adequate  performance.  A  small  error  comes  to  be  tolerated, 
and  effort  is  directed  only  at  preventing  or  correcting  large 
errors  (Helson,  1949,  p.  495).  *The  task  becomes  analogous  to 
a  vigilance  task,  and  fails  to  occupy  the  man's  full  channel 
capacity  or  attention. 

At  this  stage  the  level  of  performance  can  be  improved  by 
presenting  the  man  with  a  challenge. .. .knowledge  of  results 
can  reduce  the  size  of  the  error  which  the  man  will  tolerate, 
and  so  raise  the  standard  of  his  performance. 

Unfortunately,  a  change  in  experimental  conditions  that  makes 
the  task  harder  may  also  present  a  challenge  to  the  man.  This 
means  that  the  poorer  performance  which  is  to  be  expected  as 
a  result  of  increased  difficulty  of  the  task  may  be  partly 
offset  by  the  challenge  effect.  Tracking  in  one  dimension  is 
thus  not  as  sensitive  to  changes  in  experimental  conditions 
as  are  tasks  which  occupy  the  man’s  channel  capacity  more 
fully...  (1969,  pp.  312-313) 

Poulton' s  analysis  indicates  that  the  operator  may  decide  to  limit 
control  effort  to  the  prevention  or  correction  of  large  errors.  In  his 
view,  this  decision  converts  the  task  from  pure  tracking  to  vigilance 
performance  conditions.  Success  in  vigilance  performance  is  determined 
by  error  detection,  the  degree  of  error  to  be  tolerated,  and  skill  in 
error  correction.  Error  detection  will  reflect  differences  in  operator 
vigilance  strategy.  To  prevent  large  errors,  the  operator  maintains  a 
higher  level  of  attention  or  effort  to  anticipate  and  respond  to  perfor¬ 
mance  conditions  which,  if  uncorrected,  would  result  in  unacceptably 
large  errors.  On  the  other  hand,  when  the  operator  strategy  is  to 
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correct  large  errors,  the  operator  responds  only  if  he  has  detected  the 
occurrence  of  deviations  which  have  exceeded  his  acceptable  tolerance 
limit. 

An  operator  shift  from  pure  tracking  to  one  of  the  vigilance  perfor¬ 
mance  strategies  would  explain  how  the  adaptive  logic  in  the  Gopher  et  al. 
(1974)  testing  paradigm  allowed  subjects  to  exhibit  differential  improve¬ 
ments  over  baseline  predictions  in  dual-task  performance.  The  adaptive 
logic  in  the  Gopher  et  al.  paradigm  was  expressed  as  a  function  of  target 
error  measured  as  deviation  from  center  of  the  visual  display.  When  error 
was  consistently  less  than  10%  of  display  length,  task  difficulty  was 
progressively  increased.  If  error  consistently  exceeded  the  10%  limit, 
task  difficulty  was  reduced.  Task  difficulty  stablized  when  the  errors 
were  distributed  about  equally  above  and  below  the  limiting  value.  Given 
stable  or  increasing  levels  of  skill,  an  operator  decision  to  tolerate 
greater  error  would  cause  an  increase  in  the  observed  deviations  which 
would,  in  turn,  cause  a  decrease  in  the  existing  estimate  of  task  diffi¬ 
culty.  The  amount  of  decrease  would  be  a  direct  function  of  the  increase 
in  error  tolerance.  In  subsequent  performances  the  operator  would  be  able 
to  achieve  correspondingly  less  average  error  than  predicted  for  higher 
levels  of  difficulty  because  the  observed  estimate  of  task  difficulty  under¬ 
estimated  the  true  level  of  skill. 

Although  the  tolerance  for  error  process  invalidates  existing 
procedures  to  estimate  task  difficulty  with  an  adaptive  logic  approach, 
it  must  also  be  accounted  for  in  a  forced  difficulty  paradigm,  e.g. , 

Pew  et  al.  (1977).  Poulton's  analysis  implies  that  a  decision  to  limit 
control  effort  represents  the  end  of  a  learning  phase  in  skill  acquisition. 
However,  the  operator  might  become  bored,  fatigued,  or  otherwise  disinclined 
to  maintain  effort  to  learn  or  perform  before  completely  mastering  the  task. 
Estimates  of  task  difficulty  before  a  decision  to  switch  from  tracking  to 
vigilance  performance  would  thus  underestimate  the  true  asymptotic  level 
of  achievement.  As  an  aside,  there  would  be  some  training  management  value 
in  knowing  the  extent  of  any  skill  improvement  which  might  occur  as  a 
function  of  practice  after  the  switch  to  the  vigilance  mode  of  performance. 

The  concept  of  tolerance  for  error  and  the  corresponding  switch  from 
tracking  to  vigilance  performance  strategies  has  definite  measurable 
implications.  Suppose  performance  is  represented  as  a  sequence  of  obser¬ 
vations  of  a  measure  of  skill  from  repeated  trials  across  some  extended 
period  of  time.  If  greater  effort  in  the  learning  phase  corresponds  to 
improvement  of  skill  level  and  a  constant  or  perhaps  decreasing  level 
of  performance  variability,  data  from  the  repeated  observations  should 
exhibit  a  definite  trend  of  improvement  of  level  of  skill.  An  increased 
level  of  error  after  the  shift  to  the  vigilance  phase  should  be  observed 
as  a  discontinuity  of  either  mean  or  variability  of  performance.  In  the 
vigilance  phase,  the  observations  should  represent  random  samples  from  a 
distribution  with  mean  and  variance  determined  by  the  degree  of  error 
tolerance  and  the  particular  vigilance  performance  strategy.  Statistical 
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methods  for  estimating  parameters  from  repeated  observations  will  be  con¬ 
sidered  after  a  brief  summarization  of  the  implication  that  an  operator 
may  attempt  to  minimize  effort  rather  than  maximize  performance. 

To  summarize  the  implications  of  Poulton’s  concept  of  tolerance  for 
error,  it  was  hypothesized  that  (a)  differences  in  operator  goals,  attitudes 
and  the  like  would  be  represented  in  different  performance  strategies,  (b) 
these  strategies  could  be  operationally  defined  on  a  scale  of  performance 
effort,  and  (c)  different  strategies  and  tolerances  for  error  would  lead 
to  measurable  differences  in  patterns  of  performance  associated  with  the 
corresponding  level  of  effort.  The  two  extremes  of  the  scale  of  effort 
would  be  performance  maximization  at  the  high  effort  end  and  effort 
minimization  at  the  low  end.  Figure  1  depicts  a  schematic  layout  of  the 
scale  of  effort  concept  and  the  ordering  of  performance  strategies  which 
were  logically  differentiated  in  the  preceeding  analysis  of  the  tolerance 
for  error  concept. 

Mean  Square  Successive  Differences 

Standard  statistical  methods  from  the  area  of  time-series  data 
analysis  provided  the  analytic  tools  needed  to  evaluate  both  trend  and 
variability  components  in  a  sequence  of  tracking  performance  observations. 
Since  these  methods  are  commonly  used  in  engineering  and  economic  analyses, 
some  of  them  may  not  be  familiar  to  the  psychologist.  An  understanding  of 
mean  square  successive  differences  (MSSD)  is  crucial  to  the  interpretation 
of  the  results  of  this  investigation.  Therefore,  MSSD  is  described  in 
limited  detail  here.  Readers  interested  in  greater  detail  should  refer  to 
the  technical  sources  and  those  alreacy  familiar  with  MSSD  may  skip  to  the 
next  section  without  any  loss  of  continuity. 

Mean  square  successive  differences  is  a  measure  of  variability  of 
performance  based  on  the  order  of  the  observations  as  the  origin.  As  a 
measure  of  trend  strength  in  a  set  of  time-series  data,  e.g.,  repeated 
measures,  MSSD  derives  its  meaning  from  the  fact  that  pairs  of  adjacent 
observations  will  be  more  highly  correlated  than  will  be  pairs  of  more 
widely  separated  values.  This  sequential  dependency  of  the  observations 
on  their  order  means  that  with  a  trend  present  in  the  data,  differences 
between  pairs  of  adjacent  observations  will  be  smaller  than  when  the  data 
is  from  a  random  sample.  The  variance  is  the  average  variability  of  the 
observations  with  the  mean  as  the  origin.  Therefore,  a  comparison  of  the 
variance  with  MSSD  will  be  an  index  of  trend  strength.  When  there  is  a 
linear  or  polynomial  trend  in  the  data,  the  MSSD  will  be  small  relative 
to  the  variance  as  illustrated  in  Figure  2.  Without  a  stable  trend, 

MSSD  will  approach  the  variance  as  a  measure  of  variability.  (See 
Brownlee,  1965,  pp.  221-223  for  a  proof  and  more  detail  on  computational 
methods. ) 

Standard  methods  are  used  to  transform  the  ratio  of  MSSD  to  the 
variance  into  a  standard  normal  deviate,  i.e.,  a  z-score  (Brownlee,  1965). 

As  a  standard  normal  deviate  this  transformed  ratio  can  be  employed  to 


determine  the  departure  of  the  data  from  randomness  in  the  conventional 
statistical  way.  That  is,  the  investigator  posits  an  alpha  probability 
and  accepts  or  rejects  the  null  hypothesis  of  no  trend  as  the  obtained 
z-score  indicates.  Brownlee  reports  that  other  Investigators  have  shown 
that  the  z^-score  transform  is  acceptable  with  as  few  as  ten  observations 
and  tables  exist  for  use  with  as  few  as  four  observations.  Unfortunately, 
these  tables  are  not  generally  available  and  the  occasional  user  may  find 
it  difficult  to  obtain  copies  (see  Hart,  1942,  for  tables). 

Research  Hypothesis 

The  preceding  analyses  suggested  that  (a)  the  concept  of  tolerance 
for  error  would  associate  changes  in  performance  effort  and  differences  in 
such  attitudinal  variables  as  operator  goals ,  motivation  or  interest  in  the 
task  with  differences  in  patterns  of  performance,  particularly  variability 
of  performance,  over  time;  and,  (b)  the  MSSD  measure  would  discriminate  the 
presence  or  absence  of  trends  in  time  series  data.  Suppose  that  two  groups 
of  subjects  were  selected  on  the  basis  of  presumed  differences  in  attitude, 
that  if  present,  these  attitudinal  differences  would  result  in  differences 
in  performance  effort,  and  that  members  of  these  groups  were  given  a  series 
of  trials  with  the  Pew  et  al.  (1977)  visual  tracking  test  in  critical 
tracking  mode.  Finally,  if  the  MSSD  measure  was  then  used  .to  categorize 
performance  by  the  members  of  each  group^into  subgroups  of  random  or  non- 
random,  analysis  of  trends  or  variability  in  the  data  for  the  resulting 
two  by  two  contingency  table  should  reveal  an  interaction  of  attitudinal 
group  with  type  of  performance  across  blocks  of  performance  trials.  The 
trials  would  be  blocked  to  provide  means  and  standard  deviations  to  estimate 
the  "local"  level  of  achievement  and  variability  of  performance.  The 
following  data  collection  and  analysis  methods  were  employed  to  test  this 
hypothesis  of  a  triple  interaction. 


METHODS 


Subjects 

Data  for  this  investigation  were  obtained  from  the  records  of  29 
individuals  who  had  participated  in  a  coroo rehens ive  selection  testing 
research  program.  Nine  of  the  individuals  had  recently  resigned  or  been 
eliminated  from  warrant  officer  or  helicopter  pilot  training  and  20  of 
their  contemporaries  were  still  in  the  Army  warrant  officer  helicopter 
pilot  training  program  at  th  US  Army  Aviation  Center,  Fort  Rucker,  AL. 

Test  Apparatus 

A  model  620  Visual  Tracking  Analyzer  manufactured  by  Bolt,  Beranek 
and  Newman,  Cambridge,  MA,  was  used  to  administer  the  visual  tracking  test. 

The  model  620  is  capable  of  testing  in  either  fixed  or  critical  tracking 
mode  but  this  investigation  was  limited  to  critical  tracking  data.  The 
light  spot  is  displayed  on  a  horizontal  unit  20  by  7.5  by  10  cm  which  contains 
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a  horizontal  line  of  64  light  emitting  diodes,  each  spaced  2,54  mm  apart. 

The  display  unit  is  connected  to  a  master  control  unit  by  a  15  foot  wire 
cable  with  connectors  at  each  end.  The  master  control  unit  provides  basic 
electronic  circuitry,  power  supply,  and  the  tester's  unit.  The  tester's 
unit  provides  controls  to  (a)  select  the  mode  of  tracking  operation,  (b) 
set  the  number  of  trials  per  testing  block,  (c)  start  a  block  of  test 
trials,  (d)  enable  the  start  of  each  test  trial,  (e)  reject  any  unsuitable 
trial  performance,  and  (f)  conduct  a  standard  system  checkout  to  verify 
each  of  the  system  functions  and  displays  and  provide  demonstrations  of 
key  features  to  each  subject.  Displays  on  the  tester's  unit  provide  status 
information  about  the  state  of  the  system,  number  of  the  current  trial  in 
a  block,  and  the  score  for  both  the  most  recently  completed  trial  and  the 
current  block  average. 

The  subject  controls  the  location  of  the  light  spot  with  lateral 
movements  of  a  spring-loaded,  finger  operated  joy-stick.  One  degree  of 
stick  deflection  corresponds  to  a  movement  of  2.36  mm  on  the  visual  display. 
The  control  stick  is  mounted  on  a  metal  box  11.2  by  17.5  by  5  cm  and  it  is 
connected  to  the  visual  display  unit  by  a  6  foot  wire  cable  with  connectors 
at  each  end.  The  subject's  control  unit  also  contains  a  calibration  thumb 
wheel  and  two  trial  start  buttons,  one  button  on  either  side  of  the  control 
stick. 

To  measure  the  effective  time  delay,  the  test  apparatus  is  operated  in 
the  critical  tracking  mode.  The  value  of  the  system  time  constant  at  the 
end  of  a  trial  is  the  index  of  the  subject's  effective  time  delay  for  that 
trial.  At  the  start  of  a  trial  the  system  automatically  set  the  time 
constant  at  500  milliseconds  (ms).  As  the  trial  progresses  the  time  constant 
is  reduced  at  the  rate  of  10  ms  per  second  until  the  light  spot  has  deviated 
2.5  cm  from  the  center  of  the  display  and  at  the  rate  of  2.5  ms  per  second 
after  the  light  spot  has  exceeded  the  2.5  cm  limit.  As  the  size  of  the  time 
constant  decreases,  the  rate  of  movement  on  the  display  Increases  until  the 
subject  is  unable  to  maintain  the  light  spot  location  within  the  limits  of 
the  display.  When  the  light  spot  location  exceeds  the  limits  of  the  display, 
the  system  stops  the  trial,  displays  the  trial  score  and  the  current  value  of 
the  block  mean  effective  time  delay  on  the  tester's  display,  and  signals 
an  end  of  trial  on  the  tester's  status  display.  The  tester  must  then 
record  the  trial  score  if  it  is  desired  and  enable  a  new  trial.  The  system 
is  designed  so  that  an  attempt  to  enable  a  new  trial  at  the  end  of  a  block 
will  result  in  an  end  of  block  signal  on  the  tester's  status  display. 

Procedure 

Subjects  reported  to  a  standard  testing  location  according  to  a 
prescribed  week  long  testing  schedule.  This  testing  schedule  was  worked 
out  to  provide  continuity  of  testing  over  a  five  day  period  and  to  minimize 
the  test  activity  interference  with  routine  training.  The  second  day  of 
testing  was  used  to  give  40  trials  of  the  critical  tracking  test  in  4  blocks 
of  10  trials.  The  tester  set  the  system  to  the  system  checkout/demonstration 
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mode.  When  the  subject  reported  for  testing,  he/she  was  seated  at  a  table 
with  the  finger  operated  control  stick.  The  tester  then  read  through  the 
following  instructions: 

In  this  test  your  job  is  to  control  the  movements  of  this  light 
spot  [tester  points  to  light  spot  on  visual  display]  with  the 
control  stick  in  front  of  you.  Take  hold  of  the  stick  in  a 
comfortable  position  and  move  it  right  and  left.  Notice  that 
the  control  moves  the  light  spot  back  and  forth  on  the  display. 

Later,  when  you  start  the  test,  the  light  spot  will  move 
randomly  right  or  left  on  the  display  from  time  to  time.  As  a 
test  progresses,  the  time  between  these  random  movements  gets 
shorter  and  shorter  and  it  gets  harder  and  harder  to  control 
the  position  of  the  light  spot.  Finally,  the  light  spot  goes 
out  of  control,  off  the  end  of  the  display,  and  the  system  will 
freeze  the  light  spot  at  the  end  of  the  display.  Your  score 
will  be  the  time  between  the  random  movements  when  the  light 
spot  is  frozen. 

(Tester  note:  Set  the  system  in  CRITICAL  MODE.) 

Notice  that  the  light  spot  is  now  frozen  at  the  end  of  the 
display.  Move  the  control  stick  and  notice  that  the  light 
spot  does  not  move.  When  this  happens  that  means  the  end 
of  the  test  and  I  will  read  your  score  to  you.  To  start  a 
test  you  will  find  two  buttons  next  to  the  control  stick 
marked  "START".  After  I  say  "Ready"  you  may  push  either 
button  to  start  the  test.  When  you  release  the  button,  the 
light  spot  will  automatically  move  to  the  center  of  the 
display  and  the  test  will  start.  (Tester  demonstrates.) 

Do  you  have  any  questions? 

You  will  repeat  the  test  40  times  in  the  next  hour.  After 
each  trial  I  will  read  your  score  to  you.  The  smaller  your 
score  the  better  your  performance.  Your  objective  should 
be  to  get  the  smallest  possible  score  in  the  fewest  trials. 

To  get  a  small  score  it  is  very  important  to  keep  the  light 
spot  as  near  the  center  of  the  display  as  possible.  Do  you 
have  any  questions  on  scoring? 

Each  trial  was  followed  by  15  seconds  rest  and  there  was  a  2 
minute  rest  period  after  each  block  of  10  trials.  At  the  end  of  each 
trial,  the  tester  recorded  the  trial  score,  reported  it  orally  to  the 
subject,  timed  the  rest  interval,  enabled  the  system  for  the  next 
trial  or  block,  and  at  the  end  of  the  rest  time,  announced  "Ready"  to 
signal  the  subject  to  start  the  next  trial. 

The  subject  participated  in  fixed  difficulty  tracking  on  the  third 
and  fourth  days  of  testing  before  receiving  a  final  test  in  critical 
difficulty  tracking.  On  the  third  day  the  subject  performed  fixed 


difficulty  tracking  to  establish  levels  of  skill  for  the  time-sharing 
test  given  on  the  fourth  day.  The  time-sharing  tests,  lasting  about  30 
minutes,  consisted  of  45  trials  of  fixed  difficulty  tracking  in  3  blocks 
of  15  trials,  1  block  for  each  of  3  levels  of  tracking  test  difficulty. 
Following  the  time-sharing  tests  each  subject  received  5  trials  in 
critical  tracking  mode  as  a  final  test  of  tracking  skill. 

Data 


The  tester  recorded  the  effective  time  delay  score  for  each  of  the 
40  initial  and  the  5  final  trials  of  critical  tracking.  Recorded  on  a 
standard  form  specifically  designed  for  use  with  critical  tracking  in 
the  aviator  selection  research  program,  the  critical  tracking  scores 
were  later  transcribed  to  standard  80  column  computer  card  image  forms, 
checked  by  a  second  person,  and  keypunched  with  verification.  A  special 
FORTRAN  program  was  prepared  to  compute  means  and  standard  deviations 
for  the  9  blocks  of  5  trials  and  to  compute  the  ^-score  conversion  of 
the  MSSD  measure  from  all  the  data  in  the  first  40  trials. 

Design 

A  two-way  categorization  was  used  as  the  design  of  the  subsequent 
analyses.  The  two  categories  were  type  of  subject,  trainee  versus  attritee, 
and  type  of  performance,  random  (z-score  less  than  1.96)  and  nonrandom 
(z-score  greater  than  or  equal  to  1.96);  nonrandom  in  this  case  means 
that  the  data  contained  a  linear  or  higher  order  polynomial  trend. 

Data  Analysis 

The  first  step  in  the  data  analysis  was  to  compute  a  chi-square  to  test 
the  hypothesis  that  frequency  of  classification  of  type  of  performance 
was  not  dependent  on  student  category.  Acceptance  of  this  null  hypothesis 
of  no  dependency  would  be  used  as  evidence  for  employing  a  least  squares 
analysis  of  variance  procedure  with  the  observed  cell  frequencies  as  the 
best  estimates  of  the  proportions  in  the  population.  Rejection  of  the  null 
hypothesis  of  frequency  of  classification  would  indicate  a  need  to  employ 
methods  to  adjust  the  degrees  of  freedom  in  the  analysis  of  variance 
procedures. 

A  2  between-,  1  within-sub jects  repeated  measures  analysis  of  variance 
was  used  to  test  hypotheses  about  the  equality  of  (a)  mean  effective 
time  delay  and  (b)  the  standard  deviation  of  effective  time  delay  for  the 
five  trial  blocks.  Any  effect  in  the  chi-square  test  or  the  analyses  of 
variance  was  considered  statistically  significant  at  the  conventional  .05 
level. 


RESULTS 

The  z-score  transform  from  each  subject’s  data  was  used  to  classify 
his/her  performance  as  random  or  nonrandom.  If  the  £-score  was  less 
than  1.96  the  performance  was  classified  as  random.  Any  performance 
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with  a  z^-score  greater  than  or  equal  to  1.96  was  considered  nonrandom, 
i.e.,  the  data  contained  a  trend.  As  a  one-tailed  test,  this  rule 
would  result  in  a  Type  I  classification  error  about  2.5%  of  the  time. 
Table  2  gives  the  breakdown  of  number  of  subjects  in  each  cell  of  the  two 
by  two  student  category  by  performance  type  matrix. 

Table  2 

Breakdown  of  Number  of  Subjects 


Student 

Category 

Type  of 

Performance 

Total 

Random 

Nonrandom 

Trainee 

11 

9 

20 

Attritee 

_2 

_7 

_9 

Total 

13 

16 

29 

A  chi-square  analysis  was  used  to  determine  if  the  classification 
of  random  versus  nonrandom  performance  was  dependent  on  student  category. 
The  marginal  totals  were  used  to  define  the  expected  cell  values  because 
there  was  no  prior  reason  to  expect  a  particular  breakdown  pattern.  The 
results  of  the  chi-square  analysis  revealed  no  statistically  significant 
dependency  in  the  observed  breakdown  of  number  of  subjects  (x2  =  1.53, 

£  >.10,  1  df ) .  This  result  was  interpreted  as  evidence  for  using  a 
least  squares  analysis  of  variance  for  unequal  cell  frequencies  with 
mean  effective  time  delay  (Winer,  1971). 

Mean  Effective  Time  Delay 

The  measure  of  skill  in  the  critical  tracking  test  was  effective 
time  delay.  Means  for  each  subject  for  nine  blocks  of  five  trials  in 
the  AO  practice  and  5  final  test  trials  were  analyzed  with  analysis  of 
variance  (Table  3) .  The  hypothesis  of  an  interaction  between  student 
categoty  and  type  of  performance  across  the  nine  blocks  of  five  trials 
was  not  confirmed  by  the  mean  effective  time  delay  measure.  The  analysis 
of  variance  revealed  statistically  significant  main  effect  for  blocks  of 
trials  which  indicated  that  average  performance  had  improved  with 
practice  (Figure  3) . 


There  were  two  statistically  significant  interactions  for  the  mean 
effective  time  delay.  Student  category  interacted  with  blocks  of  trials 


Table  3 


Analysis  of  Variance  Summary  for 
Block  Means  of  Effective  Time  Delay 


Source 

df 

Mean  Square 

F-Ratio 

„2 

U) 

Total 

260 

2696.22 

Between  Subjects 

28 

12648.45 

Student  Category 

(A) 

1 

5024.53 

.43 

- 

Performance  Type 

(B) 

1 

35276.90 

2.96 

.004 

A  x  B 

1 

16242.16 

1.36 

.001 

Error 

25 

11504.52 

Within  Sub.-jects 

232 

1495.09 

Blocks  (C) 

8 

24921.2. 

43.38*** 

.470 

A  x  C 

8 

1513.15 

2.63* 

.023 

B  x  C 

8 

2314.58 

4.03** 

.043 

A  x  B  x  C 

8 

245.53 

.43 

- 

Error 

200 

574.51 

*£<  .025 
**£.  .01 
***£<  .001 


(F  =  2.63,  £  <  .025,  =  8,  200,  proportion  of  variance  =  .023).  As 

shown  in  Figure  4,  the  source  of  this  interaction  effect  was  the  larger 
effective  time  delay  means  for  the  attritees  on  the  first  three  blocks  of 
trials.  The  other  statistically  significant  interaction  was  type  of  per¬ 
formance  with  blocks  of  trials  (F  =  4.03,  £  <  .001,  jif  =  8,200,  proportion 
of  variance  =  .043).  Figure  5  shows  that  the  source  of  this  effect  was 
the  difference  in  slopes  between  the  two  types  of  performance  which 
indicates  the  greater  rate  of  learning  or  degree  of  effort  for  the  non- 
random  group. 

Variability  of  Performance 

Analysis  of  the  block  standard  deviations  supported  the  hypothesis 
that  a  measure  of  variability  of  performance  would  be  more  sensitive  to 
differences  of  degree  of  effort  than  a  measure  of  central  tendency. 

Analysis  of  variance  with  the  block  standard  deviations  confirmed  the 
hypothesis  that  student  category  would  interact  with  type  of  performance 
across  the  blocks  of  trials  and  also  revealed  other  significant  differences 
(Table  4).  Figure  6  shows  mean  standard  deviation  as  a  function  of 
student  category  and  type  of  performance  across  blocks  of  trials.  One 
striking  feature  of  these  plots  is  the  extreme  differences  in  block  to  block 
variability  of  the  two  attritee  groups  in  relation  to  the  variability  of 
the  trainees.  The  random  trainee  group  exhibits  the  least  block  to  block 
variability  and  the  nonrandom  trainee  group  gives  strong  evidence  of  improve¬ 
ment  of  variability  with  practice.  Finally,  the  equivalence  of  the  mean 
standard  eviation  on  the  final  test  for  each  of  the  four  groups  strongly 
suggests  that  factors  other  than  differences  in  level  of  tracking  skill 
are  influencing  the  performances  of  the  members  of  the  different  groups. 

Some  caution  must  be  used  in  interpreting  the  variability  of  the 
random  attritee  group  because  the  group  has  only  two  subjects.  However, 
these  two  subjects  also  have  the  greatest  total  variances  of  any  of  the 
subjects  in  the  design  matrix  (Table  5).  As  would  be  expected  from  an 
insection  of  the  group  plots  in  Figure  6,  the  size  of  the  total  variances 
in  Table  5  is  correlated  with  group  membership.  This  correlation  is 
supported  by  the  significance  of  the  between  subjects  effects  in  the 
analysis  of  variance  summary  (Table  4).  Table  6  gives  the  mean  standard 
deviations  for  each  of  the  main  effects  and  the  interaction  in  the  two  by 
two  student  category  by  type  of  performance  part  of  the  design.  Finally, 
Figure  7  shows  the  interaction  of  type  of  performance  across  blocks  of 
trials  on  mean  block  standard  deviation.  The  interesting  feature  of  this 
interaction  is  the  increasing  variability  trend  of  the  random  versus  the 
decreasing  variability  trend  of  the  nonrandom  groups.  This  difference  of 
trend  of  variability  as  a.  function  of  type  of  performance  is  strong 
support  for  the  hypothesis  that  MSSD  is  an  indicator  of  differences  in 
performance  patterns. 
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Table  4 


Analysis  of  Variance  Summary  for 
Block  Standard  Deviations  of  Effective  Time  Delay 


Source 

df 

Mean  Square 

F-Ratio 

CM 

<  3 

Total 

260 

308.54 

Between  Subjects 

28 

420.86 

Student  Category  (A) 

1 

1734.88 

6.53** 

.018 

Performance  Type  (B) 

1 

1193.61 

4.49* 

.011 

A  x  B 

1 

2213.94 

8.33*** 

.024 

Error 

25 

265.67 

Within  Subjects 

232 

294.67 

Blocks  (C) 

8 

385.25 

1.48 

.012 

A  x  C 

8 

201.35 

.77 

- 

B  x  C 

8 

524.14 

2.01* 

.026 

A  x  B  x  C 

8 

935.17 

3.59*** 

.067 

Error 

200 

260.34 

*£  <.05 
**£  <.025 
***£  <.001 
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Student  Cate 


gure  6:  Interaction  of  student  category  with  type  of  performance 
across  blocks  of  trials  on  mean  block  standard  deviation 


Table  5 


Sum  of  Block  Variances  for  each  Subject 


Student  Category 

Random 

Nonrandom 

6509.8 

4890.5 

6850.4 

5519.7 

7570.0 

6520.2 

Trainee 

10040.3 

8010.5 

10903.3 

8649.5 

11549.0 

9439.6 

12211.4 

12700.4 

14599.8 

15989.9 

17152.5 

18239.4 

19180.8 

18569.7 

21580.9 

8309.4 

29657.8 

9499.5 

Attritee 

11220.6 

13429.4 

14869.6 

16350.1 

18228.8 

Interaction  of  type  of  performance  across 
blocks  of  trials  on  mean  block  standard  deviation 


Table  6 


Mean  Block  Standard  Deviations  for  Student 


Category  by  Type  of  Performance 


Student  Category 

Type  of 

Random 

Performance 

Nonrandom 

Combined 

Trainee 

33.39 

29.16 

31.49 

Attrit ee 

47.68 

34.03 

37.06 

Combined 

35.59 

31.29 

33.22 

DISCUSSION 

The  main  hypothesis  of  this  investigation  was  that  degree  of  effort 
would  be  a  source  of  confounding  in  tracking  test  performance.  The 
results  confirmed  this  hypothesis  if  degree  of  effort  varies  with  motiva¬ 
tion  to  perform  and  differences  in  motivation  depend  on  student  category. 
The  major  source  of  this  confounding  was  differences  in  variability  of  per¬ 
formance  as  a  function  of  number  of  test  trials.  The  source  of  the  inter¬ 
action  is  most  clearly  apparent  in  the  comparison  of  type  of  performance 
with  student  category  across  the  blocks  of  trials  on  mean  block  standard 
deviation  (Figure  6).  Inspection  of  mean  effective  time  delay  Interactions 
shows  that  mean  effective  time  delay  is  correlated  with  variability  of 
performance  which  is  consistent  with  Poulton's  hypothesis  of  a  shift  in 
performance  strategy. 

A  second  hypothesis  was  that  Inadequate  practice  was  a  source  of  con¬ 
founding  in  the  measurement  of  level  of  achievement  in  previous  research. 

In  this  investigation,  level  of  achievement  is  represented  by  mean  effec¬ 
tive  time  delay  and  Figure  3  clearly  shows  a  large  improvement  of  this 
measure,  even  after  the  eighth  trial  block.  A  comparison  of  mean  effec¬ 
tive  time  delay  from  this  investigation  and  a  previous  study  by  Pew  et 
al.  (1977)  with  the  same  tracking  test  further  supports  the  hypothesis  of 
inadequate  practice.  In  the  Pew  et  al.  study  92  students  in  Air  Force 
Undergraduate  Pilot  Training  at  Williams  Air  Force  Base,  AZ,  performed 
10  trials  of  the  critical  tracking  test.  Table  7  is  a  comparison  of  the 
mean  and  standard  deviation  of  effective  time  delay  for  the  last  7  trials 
of  the  Pew  et  al.  study  with  the  means  and  standard  deviations  of  5  trials 
blocks  and  the  final  test  for  trainees  in  the  present  study.  (Trainees 
were  used  for  comparability  of  populations.)  The  important  comparisons 
in  Table  7  show  that  there  were  no  significant  differences  between  the 
Pew  et  al.  results  and  those  of  this  Investigation  on  the  first  4  blocks 
of  trials. 


Table  7 


Means  and  Standard  Deviations  of  Effective  Time  Delay  from  Pew  et  al.  (1977) 
and  Blocks  of  Trials  for  Trainees  from  the  Present  Investigation 


Pew 

Block*3 

Final 

Measure 

et  al. 

1 

2 

3 

4 

5 

6 

7 

8 

Test 

Mean 

340.  3 

344.0 

327.6 

318.5 

308.5 

300.0* 

303.1* 

288.7* 

298.9* 

250.9** 

S.D. 

52.  3 

66.3 

59.4 

59.4 

58.2 

55.3 

54.1 

58.4 

60.1 

40.0 

an  *  92 ;  ^n  «  20;  *£  <  .05;  **j><  .001 


The  results  of  this  investigation  indicate  that  mean  square  successive 
differences  (MSSD)  should  be  a  useful  statistical  tool  in  subsequent  re¬ 
search.  Although  MSSD  was  employed  in  the  present  investigation  as  a  one- 
tailed  test  to  indicate  polynomial  trends,  significant  negative  values  of 
the  z-score  derived  from  MSSD  would  indicate  that  the  data  contained  system¬ 
atic  cyclic  or  periodic  trends,  i.e.  ,  trends  describable  with  trigonometric 
functions.  This  latter  feature  makes  MSSD  especially  useful  in  the  anal¬ 
ysis  of  tracking  performance  from  continuous  control  tasks  where  periodic 
features  of  the  data  may  indicate  important  differences  in  operator  control 
behaviors.  With  a  significant  positive  or  negative  £-score  from  the  MSSD 
measure,  the  data  analyst  is  justified  in  a  detailed  search  for  the  sources 
of  the  specific  polynomial  or  periodic  trends  in  an  individual  set  of  data. 

Research  is  needed  to  establish  the  predictive  validity  of  differences 
in  patterns  of  performance  from  the  tracking  test  for  overall  success  in 
pilot  training.  Interviews  with  instructor  pilots  have  indicated  that  lack 
of  motivation  is  frequently  a  source  of  inadequate  student  progress  in 
Army  helicopter  pilot  training.  This  instructor  pilot  observation  is  sup¬ 
ported  by  two  sources  of  additional  evidence.  First,  some  50Z  of  all  attri¬ 
tion  in  the  Army  helicopter  pilot  training  program  results  from  resignations 
(Elliot  &  Joyce,  1978).  Furthermore,  motivation  was  identified  as  a  major 
factor  among  resigning  students.  Second,  an  unreported  exploratory  invest¬ 
igation  at  the  US  Army  Aviation  Center  found  a  correlation  of  .78  between 
instructor  pilot  ratings  of  basic  student  pilot  qualities,  e.g.,  motivation, 
judgment  and  the  like,  on  daily  grade  sheets  from  early  primary  training  and 
subsequent  eliminations  from  advanced  training.  This  evidence  suggests  that 
the  present  approach  may  yield  a  substantial  reduction  in  the  residual  var¬ 
iance  of  the  aviator  trainee  selection  testing  process. 
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The  approach  used  In  this  investigation  also  presents  some  Interesting 
possibilities  for  further  research  in  aviator  trainee  selection  and  manage¬ 
ment  methods.  For  example,  detailed  analyses  of  individual  performance 
trends  were  not  accomplished  in  the  present  investigation.  However,  the 
logical  analysis  of  degree  of  effort  depicted  in  Figure  1  indicates  that 
differences  in  such  trends  should  further  differentiate  among  types  of 
performance  and  the  associated  performance  strategies.  One  interesting 
hypothesis  is  that  learning  behavior,  i.e.,  performance  strategy,  in  a 
simple  tracking  test  would  predict  learning  behavior  in  more  complicated 
tasks,  i.e.,  performance  in  aircraft  control. 

Cronbach  and  Snow  (1977)  evaluate  the  hypothesis  of  prediction  from 
learning  behavior  in  these  terms: 

If  individual  differences  prove  to  be  stable  and  predictable,  one  can 
capitalize  on  findings  from  the  experiment  in  which  learning  is 
observed  only  for  a  short  time,  perhaps  on  just  one  task  or  topic. 

If  individual  differences  are  radically  altered  during  learning... 
the  short-term  experiments ...  will  not  give  practically  useful  con¬ 
clusions.  Under  this  hypothesis,  persons  who  learn  most  efficiently, 
among  a  group  all  of  whom  have  become  familiar  with  the  problem, 
would  not  generally  be  the  ones  who  learned  most  efficiently  at  the 
outset;  hence,  they  would  not  have  been  among  the  most  successful 
learners  in  a  short  experiment  (p.  126). 

The  major  issue  is  whether  attitudinal  differences  such  as  motivation  which 
are  reflected  in  the  degree  of  effort  measurement  procedures  are  relatively 
stable  characteristics  of  an  individual's  learning  behavior.  As  a  test  of 
the  Cronbach  et  al.  hypothesis  in  a  subsequent  investigation,  the  methods 
of  this  investigation  will  be  employed  to  predict  performances  of  these 
same  subjects  in  fixed  stability  tracking,  time-sharing,  and  a  job-sample 
test  administered  on  the  UH-1  flight  simulator. 
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1  USA  Armament  Cmd,  Redstone  Arsenal,  ATTN:  ATSK-TEM 
1  USA  Armament  Cmd,  Rock  Island,  ATTN:  AMSAR-TDC 
1  FAA-NAFEC,  Atlantic  City,  ATTN:  Library 
1  FAA  NAFEC,  Atlantic  City,  ATTN:  Human  Engr  Br 

1  F  AA  Aeronautical  Ctr,  Oklahoma  City,  ATTN:  AAC-44D 

2  USA  Fid  Arty  Sch,  Ft  Sill.  ATTN:  Library 
1  USA  Armor  Sch,  Ft  Knox,  ATTN:  Library 

1  USA  Armor  Sch,  Ft  Knox,  ATTN:  ATSB-DI-F 
I  USA  Armor  Sch,  Ft  Knox.  ATTN:  ATSB  DT  TP 
1  USA  Armor  Sch,  Ft  Knox,  ATTN:  ATSB  CD-AD 


2  HQUSACDEC,  Ft  Ord.  ATTN:  Library 

1  HQUSACDEC.  Ft  Ord.  ATTN:  ATEC-EX-E  Hum  Factor* 

2  USAEEC.  Ft  Benjamin  Harrison,  ATTN:  Lilxaiy 

1  USAPACDC,  Ft  Benjamin  Harrison,  ATTN:  ATCP  -HR 
1  USA  Comm-Elect  Sch,  Ft  Monmouth,  ATTN:  ATSN  -EA 
1  USAEC,  Ft  Monmouth,  ATTN:  AMSEL  CT  HOP 
1  USAEC,  Ft  Monmouth,  ATTN:  AMSEL-PA  P 
1  USAEC,  Ft  Monmouth.  ATTN:  AMSEL- SI -CB 
1  USAEC,  Ft  Monmouth,  ATTN:  C,  Feci  Dev  Br 
1  USA  Materials  Sys  Anal  Agcy,  Aberdeen.  ATTN:  AMXSY  -P 
1  Edge  wood  Arsenal.  Aberdeen.  ATTN:  SARE  A  8L  H 

1  USA  Ord  Ctr  &  Sch,  Aberdeen,  ATTN:  ATSL-TEM-C 

2  USA  Hum  Engr  Lab,  Aberdeen,  ATTN:  Library /Dir 

1  USA  Combat  Arms  Tng  Bd.  Ft  Benning,  ATTN:  Ad  Supervisor 

1  USA  Infantry  Hum  Rsch  Unit,  Ft  Benning.  ATTN:  Chief 

1  USA  Infantry  Bd,  Ft  Benning,  ATTN:  STEBC-  TE-  T 

1  USASMA.  Ft  Bliss,  ATTN:  ATSS  -LRC 

1  USA  Air  Def  Sch.  Ft  Bliss.  ATTN:  ATSA  CTD  ME 

1  USA  Air  Def  Sch.  Ft  Bliss,  ATTN:  Tech  Lib 

1  USA  Air  Def  Bd.  Ft  Bliss.  ATTN:  FILES 

1  USA  Air  Def  Bd.  Ft  Bliss,  ATTN:  STEBD-PO 

1  USA  Cmd  &  General  Stf  College,  Ft  Leavenworth,  ATTN:  Lib 

1  USA  Cmd  &  General  Stf  College,  Ft  Leavenworth,  ATTN:  ATSW-SE-  f. 

1  USA  Cmd  &  General  Stf  College,  Ft  Leavenworth.  ATTN:  Ed  Advisor 
1  USA  Combined  Arms  Cmbt  Dev  Act,  Ft  Leavenworth,  ATTN:  DepCdt 
1  USA  Combined  Arms  Cmbt  Dev  Act,  Ft  Leavenworth,  ATTN:  CCS 
1  USA  Combined  Arms  Cmbt  Dev  Act,  Ft  Leavenworth,  ATTN:  ATCASA 
1  USA  Combined  Arms  Cmbt  Dev  Act,  Ft  Leavenworth,  ATTN:  ATCACO-E 
1  USA  Combined  Arms  Cmbt  Dev  Act,  Ft  Leavenworth.  ATTN:  ATCACC--OI 
1  USAECOM.  Night  Vision  Lab,  Ft  Belvoir.  ATTN:  AMSEL-NV-SD 

3  USA  Computer  Sys  Cmd,  Ft  Belvoir,  ATTN:  Tech  Library 
1  USAMERDC,  Ft  Belvoir.  ATTN:  STSFB-DQ 

1  USA  Eng  Sch,  Ft  Belvoir,  ATTN:  Library 
1  USA  Topographic  Lab,  Ft  Belvoir,  ATTN:  ETL  TD-S 
1  USA  Topographic  Lab.  Ft  Belvoir.  ATTN:  STINFO  Center 
.1  USA  Toi>ographic  Lab,  Ft  Belvoir.  ATTN:  ETL  GSL 
I  USA  Intelligence  Ctr  &  Sch,  Ft  Hu.tchurj,  ATTN:  CTD  MS 
1  USA  Intelligence  Ctr  &  Sch,  Ft  Huachuca,  ATTN:  ATS-CTD— MS 
t  USA  Intelligence  Ctr  &  Sch,  Ft  Huachuca.  ATTN:  ATSI-TE 
1  USA  Intelligence  Ctr  &  Sch,  Ft  Huachuca,  ATTN:  ATSI-TEX  -GS 
1  USA  Intelligence  Ctr  &  Sch,  Ft  Huachuca.  ATTN:  ATSI-CTS-  OR 
1  USA  Intelligence  Ctr  &  Sch.  Ft  Huachuca.  ATTN:  ATSI-CTO  DT 
1  USA  Intelligence  Ctr  &  Sch,  Ft  Huachuca,  ATTN:  ATSI— CTO— CS 
1  USA  Intelligence  Ctr  &  Sch,  Ft  Huachuca,  ATTN:  OAS/SRD 
1  USA  Intelligence  Ctr  &  Sch,  Ft  Huachuca.  ATTN:  ATSI-TEM 
1  USA  Intelligence  Cti  &  Sch,  Ft  Huachuca,  ATTN:  Library 

1  CDR,  HQ  Ft  Huachuca.  ATTN:  Tech  Ref  Div 

2  CDR,  USA  Electronic  Prvg  Grd.  ATTN:  STEFP  MT~S 
1  HQ,  TCATA,  ATTN:  Tech  Library 

1  HQ.  TCATA.  ATTN:  AT  CAT-OP  Q,  Ft  Hood 
1  USA  Recruiting  Cmd,  Ft  Sheridan,  ATTN:  USARCPM-P 
1  Senior  Army  Adv.,  USAFAGOD/TAC,  Elgin  AF  Aux  Fid  No.  9 
1  HQ,  USARPAC,  DCSPER.  APO  SF  96556.  ATTN:  GPPE  SE 
1  Stimson  Lib,  Academy  of  Health  Sciences.  Ft  Sam  Houston 
1  Marine  Corps  Inst.,  ATTN:  Dean -MCI 
1  HQ.  USMC,  Commandant.  ATTN:  Code  MTMT 

1  HQ,  USMC,  Commandant,  ATTN:  Code  MPI -20-28 

2  USCG  Academy,  New  London,  ATTN:  Admission 
2  USCG  Academy,  New  London,  ATTN:  Library 

1  USCG  Training  Ctr,  NY.  ATTN:  CO 
1  USCG  Training  Ctr.  NY,  ATTN:  Educ  Svc  Ofc 
t  USCG,  Psychol  Res  Br.  DC.  ATTN  GP  1/62 
1  HO  Mid-Range  Br,  MC  Det.  Ouantico,  ATTN:  P8iS  Div 
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1  US  Marine  Corps  Liaison  Ofc,  AMC,  Alexandria,  ATTN  AMCGS  -f 
1  USATRAOOC,  Ft  Monroe,  ATTN:  ATRO-ED 
6  USATRAOOC.  Ft  Monroe,  ATTN  ATPR  AO 
1  USATRAOOC,  Ft  Monroe,  ATTN:  ATTS-E A 

1  USA  Forces  Cmd,  Ft  McPherson,  ATTN:  Library 

2  USA  Aviation  Test  Bd,  Ft  Rucker.  ATTN:  STEBG-PO 

1  USA  Agcy  for  Aviation  Safety.  Ft  Rucker,  ATTN:  Library 
1  USA  Agcy  tor  Aviation  Safety,  Ft  Rucker,  ATTN:  Educ  Advisor 
1  USA  Aviation  Sch,  Ft  Rucker.  ATTN:  PO  Drawer  O 

1  HQUSA  Aviation  Sys  Cmd,  St  Louis.  ATTN:  AMSAV-ZDR 

2  USA  Aviation  Sys  Test  Act.,  Edwards  AFB,  ATTN:  SAVTE-T 
1  USA  Air  Def  Sch.  Ft  Bliss,  ATTN:  ATS  A  TFM 

1  USA  Air  Muliilily  Rsch  &  Dev  Lab.  Moffett  Fid,  ATTN:  SAVDL  -  AS 
1  USA  Aviation  Sch,  Res  Tng  Mgt,  Ft  Rucker,  ATTN:  ATST -T-RTM 
1  USA  Aviation  Sch,  CO,  Ft  Rucker,  ATTN:  ATST— D-A 
t  HO.  DARCOM,  Alexandria,  ATTN:  AMXCD-TL 
1  HQ,  DARCOM,  Alexandria,  ATTN:  CDR 
1  US  Military  Academy,  West  Point,  ATTN:  Serials  Unit 
t  US  Military  Academy.  West  Point.  ATTN  -  Ofc  of  Milt  Ldrshp 
t  US  Military  Academy.  West  Point,  ATTN:  MAOR 
I  USA  Standardization  Gp.  UK.  FPO  NY.  ATTN:  MASE-GC 
•  Ofc  of  Naval  Rsch.  Arlington,  ATTN  Code  452 

3  Ofc  of  Naval  Rsch,  Arlington,  ATTN:  Code  45B 
1  Ofc  of  Naval  Rsch,  Arlington,  ATTN:  Code  450 
1  Ofc  of  Naval  Rsch,  Arlington.  ATTN:  Code  441 

1  Naval  Aerospc  Med  Res  Lab.  Pensacola,  ATTN:  Acous  Sch  Div 
l  Naval  Aerosnc  Med  Res  Lab.  Pensacola.  ATTN:  Code  LSI 
I  Naval  Aerospc  Med  Res  Lab,  Pensacola,  ATTN:  Code  L5 
I  Chief  of  NavPers,  ATTN  Pers-OR 
t  NAVAfRSTA.  Norfolk,  ATTN:  Safety  Ctr 
1  Nav  Oceanographic.  DC.  ATTN:  Code  6251,  Charts  &  Tech 
1  Center  of  Naval  Anal,  ATTN:  Doc  Ctr 
1  Nav AirSvsCom.  ATTN:  AIR  531 X 
1  Nav  BuMed,  ATTN:  713 
1  Nav  Helicopter  SubSqua  2.  FPO  SF  96601 
1  AFHRL  (FT)  Williams  AFB 
1  AFHRL  (TT)  Lowry  AFB 

1  AFHRL  (AS)  WPAFB.OH 

2  AFHRL  (DOJZ)  Brooks  AFB 

1  AFHRL  (DOJNJ  Lackland  AFB 
1  HQUSAF  (INYSD) 

1  HQUSAF  (DPXXA) 

1  AFVTG  IRD)  Randolph  AFB 

3  AMRL  (HE)  WPAFB.  OH 

2  AF  Inst  of  Tech,  WPAFB,  OH,  ATTN:  ENE/SL 
1  ATC  (XPTD)  Randolph  AFB 

1  USAF  AeroMed  Lib.  Brooks  AFB  (SUL- 4).  ATTN:  DOC  SEC 
1  AFOSR  (NL).  Arlington 

1  AF  Log  Cmd.  McClellan  AFB,  ATTN:  ALC/DPCRB 

1  Air  Foree  Academy.  CO,  ATTN:  Oept  of  Bef  Sen 
5  NavPers  6>  Dev  Ctr,  San  Diego 

2  Navy  Med  Neuropsychiatric  Rsch  Unit.  San  Diego 
1  Nav  Electronic  Lab,  San  Diego.  ATTN:  Res  Lab 

I  Nav  TrngCen,  San  Diego.  ATTN:  Code  9000-Lib 
1  NavPostGraSch.  Monterey,  ATTN:  Code  55Aa 
1  NavPostGraSch,  Monterey.  ATTN:  Code  2124 
1  NavTrngEquipCtr,  Orlando.  ATTN:  Tech  Lib 
1  US  Dept  of  Labor,  DC,  ATTN:  Manpower  Admin 
t  US  Dept  of  Justice,  DC,  ATTN:  Drug  Enforce  Admin 
1  Nat  Bur  of  Standards,  DC.  ATTN:  Computer  Info  Section 
1  Nat  Clearing  House  for  MH-Info,  Rockville 
1  Denver  Federal  Ctr,  Lakewood,  ATTN:  BLM 
1 2  Defense  Documentation  Center 

4  Dir  Psych.  Army  Hq,  Russell  Ofct,  Canberra 

I  Scientific  Advsr,  Mil  Bd,  Army  Hq,  Russell  Ofcs,  Canberra 
1  Mil  and  Air  Attache,  Austrian  Embassy 

1  Centre  de  Recherche  Des  Facteurs,  Humainede  la  Defense 
Nationale,  Brussels 

2  Canadian  Joint  Staff  Washington 

1  C/Air  Staff,  Royal  Canadian  AF,  ATTN:  Pars  Std  Anal  Br 

3  Chief,  Canadian  Def  Rsch  Staff,  ATTN:  C/CRDS(W) 

4  British  Def  Staff.  British  Embassy,  Washington 


1  Def  &  Civil  Inst  of  Enviro  Medicine,  Canada 
1  AIR  CRESS,  Kensington.  ATTN:  Info  Sys  Br 
1  Militaerpsykologisk  Tjeneste.  Copenhagen 
7  Military  Attache,  French  Embassy.  ATTN:  Doc  Sec 
1  Medecm  Chef,  C.E.R.P  A. -Arsenal,  Toulon/Naval  F  ranee 
1  Prin  Scientific  Off.  Appl  Hum  Engr  Rsch  Div,  Ministry 
of  Defense.  New  Delhi 

1  Pers  Rsch  Ofc  Library,  AKA,  Israel  Defense  Forces 
1  Minister  is  van  Defensie,  DOOP/KL  Afd  Sociaal 
Psychologische  Zaken,  The  Hague,  Netherlands 
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